Ip aliases in logical networks with hardware switches

ABSTRACT

Some embodiments provide a novel method of configuring a managed hardware forwarding element (MHFE) that implements a logical forwarding element (LFE) of a logical network to handle address resolution requests (e.g., Address Resolution Protocol (ARP) requests) for multiple addresses (e.g., IP addresses) associated with a single network interface of the logical network. The method identifies a physical port of the MHFE with which the multiple addresses are to be associated. The physical port is coupled to an end machine (e.g., a virtual machine, server, container, etc.) of the logical network. The method then modifies associations stored at the MHFE to associate the physical port with the multiple addresses.

BACKGROUND

There is a growing movement, driven by both industry and academia,towards a new network control paradigm called Software-DefinedNetworking (SDN). In Software-Defined Networking (SDN), a control planeimplements and maintains the control logic that governs the forwardingbehavior of shared network switching elements on a per user basis. Alogical network that is implemented for a tenant of a hosting system isa good example of an SDN. The virtual (logical) network of a tenant ofthe hosting system connects a set of data compute nodes (e.g., virtualmachines, etc.) that are assigned to the tenant, to each other and toother virtual and/or physical networks through a set of logical switchesand logical routers.

IP aliasing provides multiple IP addresses for a single interface in thelogical network, allowing a single machine in a logical network to bereached by multiple, different addresses. One of the challenges intoday's hosting system networks is providing ARP suppression forhardware virtual tunnel end points (VTEPs) when implementing IP aliasingfor logical interfaces in the logical network.

BRIEF SUMMARY

Some embodiments provide a novel method of configuring a managedhardware forwarding element (MHFE) that implements a logical forwardingelement (LFE) of a logical network to handle address resolution requests(e.g., Address Resolution Protocol (ARP) requests) for multipleaddresses (e.g., IP addresses) associated with a single networkinterface of the logical network. The method identifies a physical portof the MHFE with which the multiple addresses are to be associated. Thephysical port is coupled to an end machine (e.g., a virtual machine,server, container, etc.) of the logical network. The method thenmodifies associations stored at the MHFE to associate the physical portwith the multiple addresses.

Some embodiments provide a network control system that manages physicalforwarding elements (e.g., hardware/software routers and switches) toimplement logical networks. The network control system of someembodiments configures and manages physical forwarding elements of aphysical network to implement one or more logical networks for one ormore tenants of a hosting system (e.g., a datacenter or a collection ofdatacenters). The physical forwarding elements of some embodimentsinclude both software (e.g., virtual switches operating in a hypervisor)and hardware forwarding elements (e.g., top of rack switches).

In some embodiments, the MHFEs act as virtual tunnel end points (VTEPs)for the logical network, tunneling logical network data through anetwork to other VTEPs at other managed forwarding elements (hardwareand/or software), which are coupled to other end machines of the logicalnetwork. The network control system of some embodiments manages theVTEPs of the MHFEs (or hardware VTEPs) by configuring a configurationdatabase of the hardware VTEP through a configuration protocol (e.g.,OVSDB protocol). The configuration database of some embodiments isstructured according to a standardized configuration database schemathat is shared with various third party manufacturers of the MHFEs. Inthe configuration database schema, network layer addresses (e.g.,Internet Protocol (IP) addresses) are associated with a data link layeraddress (e.g., Media Access Control (MAC) address), which is associatedwith a logical port of a logical switch in the logical network.

In some embodiments, it is desirable to assign multiple network layeraddresses (i.e., alias addresses) for a single logical data link layeraddress (e.g., MAC address) in the logical network, but currentimplementations of the configuration database schema do not allow formultiple alias addresses to be associated with a single logical datalink layer address. Some embodiments work within the existingconfiguration database schema (e.g., OVSDB schema) to alleviate problemswith excessive address resolution requests or to provide complete ARPsuppression for multiple address aliases in a logical network.

The network control system of some embodiments alleviates the issueswith address resolution in conjunction with alias addresses in a logicalnetwork by weighting the various alias addresses based on networktraffic to each of the alias addresses. The network control systemprograms the highest weighted alias address into the configurationdatabase of the hardware VTEP, allowing the hardware VTEP to suppressaddress resolution requests for at least the most frequently used aliasaddress.

In some embodiments, the network control system provides ARP suppressionfor multiple alias addresses of a logical interface by associating themultiple alias addresses with the data link layer address of the logicalinterface in the configuration database. In some embodiments, thenetwork control system stores a delimited string (e.g., “192.168.1.1;192.168.1.3”) of alias addresses in the configuration database.Alternatively, or conjunctively, the network control system of someembodiments stores one or more ranges (e.g., “192.168.1.1-192.168.1.4”)of consecutive addresses in the configuration database. The hardwareVTEPs use the delimited string and/or the ranges of consecutiveaddresses to perform ARP suppression for the alias addresses.

The network control system of some embodiments modifies the existingconfiguration database schema to provide ARP suppression for aliasaddresses in a logical network. In some embodiments, the configurationdatabase schema is modified to include a new address list table forstoring alias address lists. In some embodiments, the address list tableuses a unique identifier (e.g., a Universally Unique Identifier (UUID))as a key for each address list. The unique identifier is then stored inthe existing table to point to the address list table, allowing the MHFEto perform ARP suppression based on the alias address list for eachlogical port.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all of the inventive subject matter disclosed in thisdocument. The Detailed Description that follows and the Drawings thatare referred to in the Detailed Description will further describe theembodiments described in the Summary as well as other embodiments.Accordingly, to understand all the embodiments described by thisdocument, a full review of the Summary, Detailed Description and theDrawings is needed. Moreover, the claimed subject matters are not to belimited by the illustrative details in the Summary, Detailed Descriptionand the Drawing, but rather are to be defined by the appended claims,because the claimed subject matters can be embodied in other specificforms without departing from the spirit of the subject matters.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purposes of explanation, several embodiments of theinvention are set forth in the following figures.

FIGS. 1a-b illustrate an example of ARP suppression with IP aliasing fora logical network.

FIG. 2 illustrates a network control system that configures and managesboth software forwarding elements and hardware forwarding elements toimplement one or more logical networks.

FIG. 3 conceptually illustrates an example of a logical network thatconnects a set of end machines to a physical machine.

FIG. 4 illustrates an example of how the CCP cluster of some embodimentscommunicates with several forwarding elements (hardware and software) toimplement one or more logical networks.

FIGS. 5a-b illustrate an example of the tunneling performed by the VTEPof an MHFE for packets through a logical network.

FIG. 6 conceptually illustrates a process for configuring forwardingtables of a VTEP for IP aliasing using weighted IP addresses.

FIG. 7 illustrates an example of weighting alias IP addresses toalleviate ARP requests for a MHFE.

FIG. 8 conceptually illustrates a process for configuring an interfacewith multiple IP addresses.

FIG. 9 conceptually illustrates a process for a MHFE to implement IPaliasing.

FIG. 10 illustrates an example of implementing IP aliasing usingdelimiters.

FIG. 11 illustrates an example of implementing IP aliasing using ranges.

FIG. 12 illustrates an example of implementing IP aliasing usingadditional tables.

FIG. 13 conceptually illustrates an electronic system with which someembodiments of the invention are implemented.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are set forth anddescribed. However, it should be understood that the invention is notlimited to the embodiments set forth and that the invention may bepracticed without some of the specific details and examples discussed.

Some embodiments provide a novel method of configuring a managedhardware forwarding element (MHFE) that implements a logical forwardingelement (LFE) of a logical network to handle address resolution requests(e.g., Address Resolution Protocol (ARP) requests) for multipleaddresses (e.g., IP addresses) associated with a single networkinterface of the logical network. The method identifies a physical portof the MHFE with which the multiple addresses are to be associated. Thephysical port is coupled to an end machine (e.g., a virtual machine,server, container, etc.) of the logical network. The method thenmodifies associations stored at the MHFE to associate the physical portwith the multiple addresses.

Some embodiments provide a network control system that manages physicalforwarding elements (e.g., hardware/software routers and switches) toimplement logical networks. The network control system of someembodiments configures and manages physical forwarding elements of aphysical network to implement one or more logical networks for one ormore tenants of a hosting system (e.g., a datacenter or collection ofdatacenters). The physical forwarding elements of some embodimentsinclude both software (e.g., virtual switches operating in a hypervisor)and hardware forwarding elements (e.g., top of rack switches).

In some embodiments, the MHFEs act as virtual tunnel end points (VTEPs)for the logical network, tunneling logical network data through anetwork to other VTEPs at other managed forwarding elements (hardwareand/or software), which are coupled to other end machines of the logicalnetwork. The network control system of some embodiments manages theVTEPs of the MHFEs (or hardware VTEPs) by configuring a configurationdatabase of the hardware VTEP through a configuration protocol (e.g.,OVSDB protocol). The configuration database of some embodiments isstructured according to a standardized configuration database schemathat is shared with various third party manufacturers of the MHFEs. Inthe configuration database schema, network layer addresses (e.g.,Internet Protocol (IP) addresses) are associated with a data link layeraddress (e.g., Media Access Control (MAC) address), which is associatedwith a logical port of a logical switch in the logical network.

In some embodiments, it is desirable to assign multiple network layeraddresses (i.e., alias addresses) for a single logical data link layeraddress (e.g., MAC address) in the logical network, but currentimplementations of the configuration database schema do not allow formultiple alias addresses to be associated with a single logical datalink layer address. Some embodiments work within the existingconfiguration database schema (e.g., OVSDB schema) to alleviate problemswith excessive address resolution requests or to provide complete ARPsuppression for multiple address aliases in a logical network.

The network control system of some embodiments alleviates the issueswith address resolution in conjunction with alias addresses in a logicalnetwork by weighting the various alias addresses based on networktraffic to each of the alias addresses. The network control systemprograms the highest weighted alias address into the configurationdatabase of the hardware VTEP, allowing the hardware VTEP to suppressaddress resolution requests for at least the most frequently used aliasaddress.

In some embodiments, the network control system provides ARP suppressionfor multiple alias addresses of a logical interface by associating themultiple alias addresses with the data link layer address of the logicalinterface in the configuration database. In some embodiments, thenetwork control system stores a delimited string (e.g., “192.168.1.1;192.168.1.3”) of alias addresses in the configuration database.Alternatively, or conjunctively, the network control system of someembodiments stores one or more ranges (e.g., “192.168.1.1-192.168.1.4”)of consecutive addresses in the configuration database. The hardwareVTEPs use the delimited string and/or the ranges of consecutiveaddresses to perform ARP suppression for the alias addresses.

The network control system of some embodiments modifies the existingconfiguration database schema to provide ARP suppression for aliasaddresses in a logical network. In some embodiments, the configurationdatabase schema is modified to include a new address list table forstoring alias address lists. In some embodiments, the address list tableuses a unique identifier (e.g., a Universally Unique Identifier (UUID))as a key for each address list. The unique identifier is then stored inthe existing table to point to the address list table, allowing the MHFEto perform ARP suppression based on the alias address list for eachlogical port.

FIGS. 1a-b illustrate an example of ARP suppression with IP aliasing fora logical network in six stages 101-106. In this example, several endmachines (virtual machines (VM1-VM3) and physical machines (PM1)) areconnected to each other through a logical network (not shown). Each ofthe end machines VM1-VM3 and PM1 has a data link layer address (i.e.,MAC addresses MAC1-MAC4) and one or more network layer addresses.Specifically, end machines VM1, VM3 and PM1 each have one network layeraddress (i.e., IP addresses IP1, IP3, and IP4 respectively), while VM2has multiple network layer addresses (i.e., IP addresses IP2 a-IP2 c).The data link layer and network layer addresses are used to identify aninterface (and port) for each end machine within the logical network.

The first stage 101 shows various machines (i.e., VM1-VM3 and PM1) thatcommunicate through managed forwarding elements 115, 120, and 125 of aphysical network. While shown as virtual machines (VM1-VM3) and physicalmachines (PM1) in this figure and other figures below, it should beunderstood that other types of data compute nodes (e.g., namespaces,containers, etc.) may connect to the managed forwarding elements (andthe logical network) in some embodiments.

A network control system 130 configures (as shown by the dashed lines)the managed forwarding elements 115, 120, and 125 to tunnel network dataof the end machines between the managed forwarding elements 115, 120,and 125. Each of the managed forwarding elements 115, 120, and 125operates as a virtual tunnel end point (VTEP) for the tunnels to theother managed forwarding elements. As a VTEP, the managed forwardingelements of some embodiments tunnel the network data for the machinesthrough a network (e.g., the Internet) by encapsulating anddecapsulating the network data with overlay network information.Although in the illustrated example, as well as other examples below,only one MHFE and two MSFEs are shown, one of ordinary skill in the artwould realize that any number of MHFEs and MSFEs can connect to thelogical network (or multiple logical networks) in the same manner.

In this example, the managed forwarding elements 115, 120, and 125include both managed software forwarding elements (MSFEs) and managedhardware forwarding elements (MHFEs). The MSFEs of some embodiments arevirtual switches (e.g., Open vSwitch, Elastic Sky X (ESX), etc.) thatrun within the hypervisor of host machines and connect to end machinesoperating on the host machine, such as virtual machines and/orcontainers. The MHFEs of some embodiments are hardware forwardingelements (e.g., a third-party hardware switch such as a top-of-rack(TOR) switch) that connect to machines (e.g., third-party serversconnected to a TOR switch, VMs operating on a server without a virtualswitch that performs logical network processing, etc.) to exchangenetwork data with other end machines and/or external networks that areconnected to the logical network.

In the second stage 102, physical machine PM1 needs to send a datamessage to network layer address IP2 b for virtual machine VM2, but doesnot have a data link layer address for virtual machine VM2. Physicalmachine PM1 needs a data link layer address for virtual machine VM2because, in the logical network, physical machine PM1 and virtualmachine VM2 are connected to the same logical forwarding element.

The second stage 102 shows that physical machine PM1 sends an addressresolution request 150 to MHFE 125 to retrieve the L2 address for aliasaddress IP2 b of virtual machine VM2. In some embodiments, the MHFE 125then determines whether it is able to respond to the address resolutionrequest.

In many implementations, the MHFEs use a standardized database schemathat allows a controller (e.g., local controller) to specify a single L3address for each port (and/or L2 address) of the logical forwardingelements. When the network control system 130 specifies a L2 address foran address resolution request, the MHFE 125 is able to perform ARPsuppression, also known as ARP proxy, handling the address resolutionrequests by crafting an ARP reply packet with the data link layeraddress of the requested network layer addresses from the logicalnetwork. By crafting the reply packet directly, the MHFE 125 avoidshaving to broadcast (or multicast) the ARP request to all of the otherlogical ports of the logical switch (i.e., to all of the MFEs thatimplement the logical ports of the logical switch).

However, in the current standardized database schema, certain issues mayarise when IP aliasing is implemented for ports and interfaces of thelogical network. IP aliasing assigns multiple network layer addresses toa single port or interface in the logical network, and allows an endmachine to be reached by any of the multiple network layer addresses. Inthis example, the network layer address IP2 b is one of three aliasaddresses (IP2 a-IP2 c) for virtual machine VM2. In some cases, thedifferent addresses are backup addresses or associated with differentservices of the end machine. With IP aliasing, the data link layeraddress of each interface remains the same, so even if multiple networklayer addresses are assigned to an interface, layer 2 connectivity wouldcontinue to work.

However, in the logical network, address resolution using the currentconfiguration database schema would be suboptimal because only a singleaddress can be specified for each interface (or port) on the logicalnetwork. ARP suppression, as described above, would only work for asingle network layer address. The MHFE 125 would have to broadcast ormulticast all ARP requests for any of the other alias addresses.

In some embodiments, the network control system 130 works to alleviatethe problem of excess ARP requests by configuring the MHFE 125 to handlea primary address (i.e., a most heavily weighted address) for eachaliased port (or interface) based on network traffic to the differentalias addresses. In other embodiments, the network control system 130configures the MHFE 125 to handle multiple alias addresses for eachinterface by associating multiple addresses with the aliased logicalport. These embodiments are described in further detail below inSections II and III.

Once the MHFE 125 determines that it is able to handle the addressresolution request, the third stage 103 shows that MHFE 125 crafts andsends back an ARP reply 155 with the MAC address MAC2 of VM2, whichcorresponds to the desired IP IP2 b.

In the fourth stage 104, the physical machine PM1 sends a data message160 destined to virtual machine VM2 with an alias address IP2 b and theMAC address MAC2. The data message 160 indicates both the network layeraddress (IP2 b) and the L2 address (MAC2) for the destination machineVM2, sending the packet as it would if PM1 were connected to a sameforwarding element as VM2.

However, in order to implement the logical forwarding element, the MHFE125 must then encapsulate the data message 160 to tunnel it to the VTEPof the destination end machine VM2. In some embodiments, the header(tunnel) information is configured by the network control system 130,which distributes configuration data to the MSFEs 115 and 120 and MHFE125. For instance, the configuration data of some embodiments specifiesthe location (e.g., IP address) of each VTEP for each port of thelogical forwarding elements associated with each managed forwardingelement. In some embodiments, each VTEP (or MHFE) uses ARP requests todiscover and maintain the network layer addresses for the other VTEPs.

The MHFE 125 of some embodiments uses configuration data received fromthe network control system 130 to identify the corresponding networklayer address (IP6) for MSFE 120, which connects to the destination endmachine VM2. The fifth stage 105 shows that the MHFE 125 takes the datamessage 160 and encapsulates it with a new header that includes thenetwork layer address (IP6) for MSFE 120, and sends the encapsulateddata message 165 to the VTEP (MSFE 120) for the destination end machineVM2.

Finally, in the sixth stage 106, the destination MSFE 120 removes thetunnel header and sends the decapsulated data message 160 to thedestination end machine VM2.

An overview of the process for implementing IP aliasing for a logicalnetwork through MHFEs has been described above. Further details andexamples of implementing IP aliasing are described below. Specifically,Section II describes using weighting to alleviate issues with IPaliasing in a logical network. Section III then describes examples ofproviding ARP suppression for IP aliases in a logical network. SectionIV then describes an electronic system with which some embodiments ofthe invention are implemented. However, before describing theseadditional examples, the environment in which some embodiments of theinvention are implemented will be described below in Section I.

I. Environment

The following section will describe the environment in which someembodiments of the invention are implemented. In some embodiments, alogical network of the hosting system defines a set of logicalforwarding elements (e.g., logical switches and routers) that logicallyconnects a group of end machines (e.g., virtual machines, containers,servers, etc.) and are implemented on resources of a physical network.The relationship between the logical and physical networks will bedescribed in greater detail below with reference to FIG. 3.

FIG. 2 illustrates an example of a network control system of someembodiments that configures and manages both software forwardingelements and hardware forwarding elements to implement one or morelogical networks (e.g., for one or more tenants of a hosting system(e.g., a datacenter or collection of datacenters)). The figure shows anetwork control system 200, host machines 220 and 225, and a third-partyrack 230.

The network control system 200 of some embodiments includes a set ofcontrollers that perform various functions for implementing the logicalnetwork. Although shown as separate from the host machines 220 and 225,in some embodiments the network control system 200 includes controllersthat operate within the host machines 220 and 225 and/or within the MHFE270 of the third-party rack 230. Some embodiments of the network controlsystem are described in further detail below.

Host machines 220 and 225 each include a managed software forwardingelement (MSFE) 245 and a set of compute nodes (e.g., end machines) 240(in this example, virtual machines). The MSFE 245 forwards logicalnetwork data for the set of compute nodes through the network. In someembodiments, the MSFE 245 is implemented in virtualization software thatoperates on the host machine.

The third-party rack 230 includes a managed hardware forwarding element(MHFE) 270, and a set of servers 280 that are connected to the MHFE 270.The MHFE 270 is a hardware device that forwards logical network data forthe set of servers 280 through the network. In some embodiments, theservers 280 include host machines for VMs that operate on legacyhypervisors of the host machines. The legacy hypervisors of someembodiments do not support logical networks and/or tunneling. In somesuch embodiments, the legacy hypervisor operating on a host machine(i.e., one of the servers 280) sends the network traffic for the VMsoperating on the host machine to the MHFE 270 to perform tunneling andlogical processing for the VMs.

In some embodiments, the managed forwarding elements MSFEs 245 and MHFE270 tunnel network data between themselves, serving as virtual tunnelend points (VTEPs) that encapsulate and decapsulate the logical networkdata of the end machines to be sent between the VTEPs. Software VTEPs ofsome embodiments enable compute nodes operating on host machines tocommunicate through tunnels with other end machines and/or networks on alogical network. Hardware VTEPs enable the physical workloads connectedto the MHFE (e.g., third-party servers connected to a TOR switch) toexchange network data with other end machines and/or external networksthat are connected to other software/hardware forwarding elements of thelogical network. In some embodiments, the VTEPs directly couple to atleast one of the end machines of the logical network.

The network control system 200 of some embodiments manages andconfigures both software and hardware VTEPs to create the tunnels (e.g.,a Virtual Extensible LAN (VXLAN) tunnel, a Stateless Transport Tunneling(STT) tunnel, etc.) between the VTEPs, using local controllers that eachmanage one of the VTEPs at the managed forwarding elements. In someembodiments, the local controllers for software and hardware VTEPs usedifferent protocols (e.g., NETCPA protocol, Open vSwitch DatabaseManagement (OVSDB) protocol, OpenFlow protocol, etc.) and configurationdatabase schemas for the software and hardware forwarding elements, orfor different functions (e.g., forwarding and tunnel creation) performedfor the forwarding elements. In some embodiments, the local controllersfor a hardware VTEP uses a common configuration database schema (e.g.,OVSDB schema) to control and configure the hardware VTEP devices.

One of ordinary skill in the art would realize that the number of thehost machines, third-party racks, and MHFEs illustrated in the figureare exemplary and a logical network for a tenant of a hosting system mayspan a multitude of host machines and third-party devices, and logicallyconnect a large number of end machines to each other and to severalother third-party devices. Also, one of ordinary skill in the art wouldrealize that even though the machines that are connected to the MHFE inthis example are servers in a third-party rack, any other third-partydevice (e.g., computer, printer, etc.) could be connected to the logicalnetworks through the MHFE.

In some embodiments, the network control system 200 receives (e.g.,through an application programming interface (API)) a definition of alogical forwarding element of the logical network from a user (e.g., anetwork administrator, a tenant of a datacenter, etc.) of the networkcontrol system 200. The network control system 200 then implementslogical networks on managed forwarding elements of a physical network byconfiguring managed forwarding elements (i.e., MSFEs 245 and MHFE 270)of the physical network to forward network data between the end machines(e.g., VMs and servers) attached to each of the managed forwardingelements according to the forwarding logic of the set of logicalforwarding elements. In some embodiments, the network control system 200assigns a network layer (e.g., Internet Protocol (IP)) address and adata link layer (e.g., Media Access Control (MAC)) address for each ofthe logical ports of the logical forwarding element.

A logical network logically connects a set of end machines (e.g.,virtual machines, physical servers, containers, etc.) and a set ofphysical machines (and other resources of the physical network) using aset of logical forwarding elements (e.g., logical L2 and L3 switches).This allows the physical resources of a physical network to be allocatedand shared while maintaining a logical separation between the endmachines of the different logical networks.

FIG. 3 conceptually illustrates an example of a logical network that isimplemented on a physical network. This figure shows end machines (VMs1-4 and PM5) that are logically connected according to a logical network300, while being physically connected according to a physical network305.

The logical network 300 includes three logical forwarding elements (alogical router 325 and two logical switches 320 and 330) that describethe desired forwarding behaviors for the end machines VMs 1-4 in thelogical network. In some embodiments, the virtual machines VMs 1-4operate on virtualization software that executes on the host machines.Specifically, VMs 1 and 2 are connected through logical ports to a firstlogical switch 320. VMs 3 and 4 and physical machine PM 5 is connectedthrough logical ports of a second logical switch 330. The two logicalswitches 320 and 330 are connected to ports of a logical router 325.

The physical network 305 shows the actual physical connections betweenthe end machines VMs 1-4 and PM 5. Physical network 305 shows two hosts365 and 370. The hosts 365 and 370 have MSFEs 355 and 360, which connectto VMs 1 and 3, and VMs 2 and 4, respectively. Physical network 305 alsoshows a MHFE (a hardware forwarding element) and a physical machine PM5(e.g., server, bare-metal machine, etc.) that is attached to a physicalport of the MHFE 375. The MSFEs 355 and 360 and the MHFE 375 areconnected to each other by tunnels through a network 399. Theconnections of the end machines to the logical switches as well as theconnections of the logical switches to the logical router are definedusing logical ports, which are mapped to the physical ports of the MSFEs355 and 360 and of the MHFE 375.

The connections in the logical network differ from the connections ofthe physical network. For example, although end machines VMs 3 and 4 andPM 5 are connected to a single logical switch 325 in the logical network300, they are all physically connected to different managed forwardingelements in the physical network 305. The MSFEs and MHFEs could residein different machines, different subnets, or even different datacentersin different parts of the world. The managed forwarding elementsimplement the logical network by creating tunnels for communicatinglogical network data to other forwarding elements. Establishing tunnelsthat extend the logical network overlay among MSFEs and other physicalswitches is described in more detail below by reference to FIG. 4.

The network control system of some embodiments configures the forwardingelements of the physical network to implement the logical forwardingelements of the logical network by creating tunnels and modifyingforwarding behaviors of the managed forwarding elements. In someembodiments, the MSFEs 355 and 360 and the MHFE 365 of the physicalnetwork implement the LFEs 320-230 through a set of flow entries forrouting the packets of the logical network. Alternatively, orconjunctively, some or all of the managed forwarding elements are notflow-based software forwarding elements, but instead process packetsbased on configuration data that is generated by their respective localcontrollers.

Although, in the illustrated example, each managed forwarding elementimplements logical forwarding elements for a single logical network, insome embodiments, a managed forwarding element may be attached tomachines for logical forwarding elements of different logical network. Asingle managed forwarding element can be configured to implementdifferent logical networks for different tenants.

FIG. 4 illustrates an example of how the network control system of someembodiments communicates with several forwarding elements (hardware andsoftware) 415-425 to implement one or more logical networks.Specifically, this figure shows a network control system 400 exchanginglogical forwarding and configuration data with the forwarding elementsto implement one or more logical forwarding elements and create tunnelsbetween the various software and physical forwarding elements.

The network control system 400 of some embodiments includes a centralcontroller 480 (or a cluster of central controllers), which manageslocal controllers 460, 465, and 470. The local controllers 460, 465, and470 manage MSFEs 415 and 420 and MHFE 425 respectively. In someembodiments, the central controller 480 manages the logical network andcommunicates logical forwarding data to the local controllers 460, 465,and 470, which converts the logical forwarding data to physicalforwarding data to configure the forwarding behaviors of thecorresponding managed forwarding element. In some embodiments, thephysical forwarding data includes data to encapsulate these packets intunnels in order to send the packets to other managed forwardingelements.

In some embodiments, the local controllers are a separate softwareprocess from the central controller of the network control system. Indifferent embodiments, a local controller may operate on the samemachine as one of the central controllers, or on a different machine(e.g., in a separate virtual machine, on a separate physical device,etc.).

The local controller 470 for MHFE 425 of some embodiments communicateswith the MHFE 425 over the OVSDB protocol to exchange forwarding state(e.g., L2 and/or L3 forwarding state). For instance, the MHFE 425 mightsend an update notification to the network control system (through thelocal controllers to the central controller) regarding a learned MACaddress of a machine (e.g., desktop computer, laptop) that is connectedto its port. The central controller 480 can then compute the necessarylogical data and push this logical data down to the local controllers415 and 420 for the MSFE. The MHFE 425 may also send statisticalinformation regarding traffic, bandwidth, etc. for the various ports ofthe MHFE.

The MHFE 425 also receives configuration information from the localcontroller 425. For instance, the MHFE 425 might receive the MACaddresses of the machines 430 and 440 that are coupled to the MSFEs 415and 420. The MHFE 425 of some such embodiments then calculates its ownforwarding data based on the forwarding information it receives from theCCP cluster. The method of computation of the forwarding data by aphysical switch can vary from one switch vendor to another.

In some embodiments, in addition to receiving the MAC addresses, each ofthe MHFEs serves as a tunnel endpoint in some embodiments, and receivesaddresses (e.g., IP addresses) of the other tunnel endpoints, as well asother information to use when encapsulating packets in the tunnels(e.g., logical network and logical port identifiers, etc.). In someembodiments, the MHFE 425 stores the tunnel endpoint addresses of theother MSFEs and MHFEs that implements the logical forwarding elementsand their different logical ports in a particular tunnel endpointlocator table. The tunnel endpoint locator table is one of severaldatabase tables that are configured on the MHFE through a databaseschema (e.g., OVSDB) by the network control system. The MHFE 425 usesthe received configuration data to locate other tunnel endpoints(through their IP addresses) and establish the tunnels between the TORswitch and the other endpoints.

FIGS. 5a-b illustrate an example of the tunneling performed by the VTEPof an MHFE for packets through a logical network in six stages 501-506.This example shows two managed forwarding elements (MFEs) 520 and 525with virtual tunnel end points VTEPs 1 and 2 respectively. Each VTEPshows a tunneling module 560 and a forwarder 565. Machines M1 and M2 arecoupled to MFE 520, while machine M3 is coupled to MFE 525. The NIFEs520 and 525 communicate logical network data through tunnels in thenetwork.

The first stage 501 shows that machine M2 sends a data message 550through port 2 of the MHFE 520 with a destination address MAC2. When thedata message 550 is destined for another machine that is coupled to theMHFE 520, the forwarder 565 forwards the data message directly to thedestination machine, without encapsulating the data message.

In this example, the forwarder 565 determines that the destinationmachine for the data message 550 is not located on the same MHFE 520. Insome embodiments, the forwarder 565 determines whether a destinationaddress is located on the same MHFE 520 based on a set of databaseconfiguration tables (not shown) that provide mappings between thelogical and physical networks. The mappings of some embodiments identifythe VTEP to which each logical port is connected. So, for example, ifthe data message 550 had a destination address of MAC1, whichcorresponds to the physical port of machine M1 on MHFE 520, then theforwarder 565 would send the packet directly to machine M1.

The second stage 502 shows that the forwarder 565 sends the data message550 to a tunneling module 560, which performs tunneling for datamessages through the network. In the second stage 502, the tunnelingmodule performs a lookup on the configuration database tables toidentify a location (i.e., the IP address) of the remote VTEP VTEP2 forthe destination end machine M3, and encapsulates the data message 550with headers that direct the data message 550 to the remote VTEP2.

The third stage 503 shows that the tunneling module 560 of VTEP1 hasattached a new header, which indicates the network layer address (VTEP2)for the destination VTEP (VTEP2) of the MFE 525 that is attached to thedestination machine M3. In the fourth stage 504, MFE 520 sendsencapsulated data message 555 through the network to the destinationVTEP (VTEP2). The destination VTEP receives the encapsulated datamessage 555 through port 3 of the MFE 525.

The fifth stage 505 shows that the tunneling module 560 of thedestination VTEP2 removes the header from the encapsulated data message555 and sends the decapsulated data message 550 to the forwarder 565.Finally, in the sixth stage 506, the forwarder 565 sends the datamessage 550 to the destination machine M3.

II. Weighting IP Aliases to Reduce ARP Requests

As described above, data messages for machines on the logical networkare forwarded through the logical network based on addresses, such as IPand MAC addresses. In some embodiments, it is desirable to assignmultiple IP addresses to a single interface for a machine. However,current implementations do not allow for attributing multiple addressesto a single interface for a hardware virtual tunnel end point (VTEP). Ifmultiple IP addresses are assigned, but the hardware VTEP is onlycapable of handling a single address, the physical machine may generateexcessive ARP requests. Each address that is not stored in theconfiguration database tables (or forwarding tables) of the VTEP willhave no corresponding MAC address, and will require the MHFE tobroadcast a new ARP request on the network.

Some embodiments of the invention alleviate the problem of excessive ARPrequests. Not all IP aliases are used with the same frequency. Forexample, a logical interface may have an IP address that is primarilyused, while other aliases are used as backups or for secondary services.The network control system of some embodiments applies a heuristic or analgorithm to associate weight to each of these IP addresses, and toidentify a primary alias address to be configured into the configurationdatabase tables (or VTEP forwarding tables).

FIG. 6 conceptually illustrates a process 600 for configuringconfiguration database tables (or VTEP forwarding tables) of a hardwareVTEP to handle IP aliasing using weighted IP addresses. In someembodiments, the process 600 is performed by a controller that monitorsthe relative frequency of traffic to different alias addresses in thedata plane, and associates weights with them, relative to one another.

As described with reference to FIG. 4, the local controllers of thenetwork control system, in addition to configuring, receive statisticsand other physical data (e.g., physical port status, etc.). In someembodiments, the central controllers and the local controllers use thestatistics to weight the different alias addresses assigned to a logicalport, based on the traffic destined for each alias address.

The process 600 monitors (at 605) the traffic for alias addressesassociated with a particular interface. The process 600 of someembodiments monitors the traffic by periodically sampling the number ofdata messages sent to each alias address for each logical port.

The process 600 then performs (at 610) a weighting for each of the aliasaddresses based on the monitored traffic. In some embodiments, theweighting is a simple weighting calculation that assigns the weightbased on the number of packets destined for each alias address. In someembodiments, the weighting is performed periodically, sampling a set ofpackets that are received for a particular duration of time. Othermethods of weighting account for historical patterns, time of day, etc.

The process 600 then determines (at 615) whether the highest weightedalias address is already mapped to the particular interface. When theprocess 600 determines (at 615) that the highest weighted alias addressis already mapped to the particular interface, the process 600 ends.

When the process 600 determines (at 615) that the highest weighted aliasaddress is not mapped to the particular interface, the process 600configures (at 620) forwarding tables of the VTEP to map the identifiedalias address to the particular interface. The process 600 then ends.

FIG. 7 illustrates an example of weighting alias IP addresses toalleviate ARP requests for a MHFE in three stages 701-703. The firststage 701 shows a controller 720 (e.g., central controller or localcontroller of the network control system) that manages MHFE 725. MHFE725 is coupled to physical machines PM1 and PM2 through ports 1 and 2 ofMHFE 725.

The controller 720 includes a weighting module 730 and monitoring table735. The monitoring table 735 shows, for each MAC address managed by thecontroller 720, the associated alias addresses and the number of packetssent to each alias address. In this example, the monitoring table 735shows that data link address MAC2 has three alias addresses IP2-1P4. Inthe first stage 701, no packets have yet been monitored by thecontroller 720.

The MHFE 725 includes a logical mapping table 740. The logical mappingtable 740 stores the mappings of the different MAC and IP addresses ofthe logical network, as well as the corresponding logical switch foreach. The logical mapping table 740 of some embodiments is used toprovide ARP suppression for the logical network.

In some embodiments, the logical mapping table 740 is one of manyconfiguration database tables (or VTEP forwarding tables) stored in thehardware VTEPs. In some embodiments, the configuration database isstructured according to a standardized schema that allows a networkcontrol system to configure the tables of the VTEP.

The second stage 702 shows that the controller 720 monitors trafficbetween the MHFE 725 and the other managed forwarding elements (notshown) of the network. The controller 720 counts the packets to eachalias address (IP1-IP4). The weighting module 730 then uses themonitored traffic to weight the different alias addresses and toidentify a primary alias for each aliased MAC address. In this example,of the three alias addresses (IP2-IP4) assigned to MAC2, IP3 hasreceived the most traffic (1000 packets). In some embodiments, theweighting module 730 performs a simple weighting that assigns thehighest value to the IP address with the most received packets.

The third stage 703 shows that the weighting module has selected IP3 asthe primary address for MAC2, and has modified the logical mapping table730 to associate the primary address IP3 with MAC2.

By modifying the logical mapping table 730 with the primary address IP3,the system should be able to minimize the number of ARP requests sent tothe machine with L2 address MAC2, alleviating some of the excess ARPrequests propagated through the system. In addition, this solution doesnot require any changes to the schema or to the hardware VTEPs, but thelesser used alias addresses would still require the ARP mechanism.

III. ARP Suppression by MHFEs for a Logical Network

In some embodiments, rather than simply alleviating the problem, thecontroller modifies the MHFEs to eliminate the problem altogether. FIG.8 conceptually illustrates a process for assigning multiple IP addressesto a single interface for a MHFE. In some embodiments, the process 800is performed by a controller of the network control system. The process800 begins by identifying (at 805) multiple alias addresses to beassigned to a particular interface in the logical network. In someembodiments, the alias addresses are assigned by a user (e.g.,administrator, tenant, etc.) of the network control system.

The process 800 also identifies (at 810) the particular VTEP associatedwith the particular logical interface. In some embodiments, the process800 performs a lookup on mappings of logical interfaces with physicalports to identify the particular VTEP for the managed forwarding elementthat is connected to the logical interface.

The process 800 then creates (at 815) a configuration message toconfigure forwarding tables to map the alias addresses to the particularinterface in the logical network. Various examples of configurations ofthe forwarding tables are described below with reference to FIGS. 10-12.

The process 800 then sends the configuration message to the particularVTEP (or MHFE) to configure the VTEP to properly handle the multiple IPaliases for the particular interface. In some embodiments, the process800 uses a standardized configuration database protocol that is used bymanufacturers of third-party forwarding elements (e.g., OVSDB).

Once the configuration message is sent to the particular VTEP, the VTEP(or MHFE) configures itself to handle multiple alias addresses. FIG. 9conceptually illustrates a process for a MHFE to implement IP aliasing.The process 900 receives (at 905) configuration information withmultiple IP addresses for a particular interface in the logical network.The received configuration information of some embodiments is generatedby a controller (e.g., central controller, local controller, etc.) ofthe network control system according to the process as described abovewith reference to FIG. 8.

The process 900 then modifies (at 910) the forwarding behaviors of theMHFE according to the received configuration information. In someembodiments, the process 900 modifies (at 910) the forwarding behaviorsof the MHFE by modifying a set of mapping tables (or forwarding tables)in the VTEP. The mapping tables describe mappings between logical portsof the logical forwarding elements, as well as data link layer andnetwork layer addresses for the machines attached to the logical ports.The mapping tables of some embodiments also include mappings to physicalports of the managed forwarding elements. In some embodiments, modifyingthe forwarding behaviors includes modifying the mapping tables toperform ARP suppression for the logical interfaces at the hardwareVTEPs. Examples of such modifications are described with reference toFIGS. 10-12.

FIGS. 10-12 illustrate examples of three different methods forconfiguring hardware VTEPs to provide ARP suppression for aliasedaddresses for an interface in a logical network. FIG. 10 illustrates anexample of implementing IP aliasing using delimiters in three stages1001-1003. In some embodiments, the MAC addresses are for remotemachines in the logical network.

The first stage 1001 shows that controller 1020 receives input 1050. Theinput 1050 shows that data link address MAC2 is to be assigned threenetwork layer addresses IP1-IP3. In some embodiments, the input 1050 isreceived from a user (e.g., a network administrator, tenant etc.) of thesystem through a user interface for the controller 1020.

In the second stage 1002, the controller 1020 then processes the input1050 to create a configuration message 1055. In some embodiments, thecontroller 1020 uses a configuration protocol (e.g., OVSDB) that is usedfor configuring databases of hardware and software managed forwardingelements to communicate the configuration message 1055 to the MHFE 1025.

The third stage 1003 shows that “IP” column of the VTEP forwarding table1060 for data link layer address MAC2 has been modified to show adelimited list (“IP2; IP3; IP4”) of network layer addresses IP2-1P4. Insome embodiments, the delimited list uses a special character (e.g.,“;”) that is not used in network layer addresses to separate thedifferent network layer addresses assigned to the particular L2 address.By using a single delimited list, the schema for the VTEP forwardingtable 1060 does not need to be modified at all.

In some embodiments, the delimited list is included as a part of theconfiguration message 1055, which the MHFE 1025 stores in the VTEPforwarding table 1060. Alternatively, in some embodiments, theconfiguration message 1055 simply includes a listing of the addresses,and the MHFE 1025 is configured to generate a delimited list based onconfiguration message 1055 and to store the generated delimited list inthe VTEP forwarding table 1060.

FIG. 11 illustrates an example of implementing IP aliasing using ranges.Like FIG. 10, the first stages 1101 and 1102 show that the controller1120 receives input 1150 and processes the input 1150 to create aconfiguration message 1155. However, rather than a delimited list, thethird stage 1103 shows that the VTEP forwarding table 1160 has beenmodified based on the configuration message 1155 to describe an addressrange. The address range provides a shorthand description (e.g.,“192.168.1.1-192.168.1.4”, etc.) to indicate all addresses between afirst and last address. The address range can include many moreaddresses than a delimited address list in less space. However, theaddress range is only useful for consecutive groups of addresses.Although the examples of FIGS. 10 and 11 are shown separately, it wouldbe clear to one skilled in the art that both methods could be usedtogether to create a delimited list of both addresses and address ranges(e.g., “IP2-IP4;IP7;IP8-IP11”).

FIG. 12 illustrates an example of implementing IP aliasing usingadditional tables. Similar to FIGS. 10 and 11, the first two stages 1201and 1202 show that the controller 1220 receives input 1250 and processesthe input 1250 to create a configuration message 1255. However, unlikethe previous examples, the example of FIG. 12 shows, in addition to theVTEP forwarding table 1260, an alias address table 1265. The aliasaddress table 1265 has an ID column and an IP column for storing networklayer addresses.

Once the controller 1220 sends the configuration message 1255 to theMHFE 1225, the MHFE 1225 modifies configuration tables 1260 and 1265 tomake associations between the alias network layer addresses and the L2address MAC2. In this example, the MHFE 1225 replaces the network layeraddress stored in Alias Address List (AAL) column of the VTEP forwardingtable 1260 with a unique ID (e.g., UUID) for the interface. Each recordin the alias address table 1265 stores a set of alias addresses in theAAL column for each UUID (which corresponds to a MAC address in the VTEPforwarding table 1260). The set of alias addresses may be stored in theform of address ranges, delimited lists, or a combination of both.

In some embodiments, rather than a single record for the aliasaddresses, the MHFE 1225 creates separate records in the alias addresstable 1265 using the unique ID as a key for each of the alias addresses.

In some embodiments, all of the alias addresses in the VTEP forwardingtable 1260 are replaced with unique IDs that point to entries in thealias address table 1265. Alternatively, in some embodiments, only theL3 addresses that implement IP aliasing (i.e., have multiple aliasaddresses) are replaced with unique IDs, while the remaining entriesmaintain a single network layer address for the MAC address.

Finally, referring back to FIG. 9, once the VTEP forwarding tables havebeen configured with the alias addresses, the process 900 then forwards(at 915) data messages through tunnels of the logical network based onthe modified forwarding tables of the VTEP. The modified forwardingbehaviors allow the MHFE to perform ARP suppression for all aliasaddresses for each logical interface, avoiding unnecessary ARP requests.

In some embodiments, the weighting described in Section II is used inconjunction with the various methods described in Section III. Forexample, in some embodiments, the new columns that are added to theconfiguration database schema are weighted according to the frequency ofuse and the addresses in the address lists are ordered, or givenpriority, based on the weightings.

IV. System

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or morecomputational or processing unit(s) (e.g., one or more processors, coresof processors, or other processing units), they cause the processingunit(s) to perform the actions indicated in the instructions. Examplesof computer readable media include, but are not limited to, CD-ROMs,flash drives, random access memory (RAM) chips, hard drives, erasableprogrammable read-only memories (EPROMs), electrically erasableprogrammable read-only memories (EEPROMs), etc. The computer readablemedia does not include carrier waves and electronic signals passingwirelessly or over wired connections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storage,which can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs. Finally, any combination ofseparate programs that together implement a software invention describedhere is within the scope of the invention. In some embodiments, thesoftware programs, when installed to operate on one or more electronicsystems, define one or more specific machine implementations thatexecute and perform the operations of the software programs.

FIG. 13 conceptually illustrates an electronic system 1300 with whichsome embodiments of the invention are implemented. The electronic system1300 may be a computer (e.g., a desktop computer, personal computer,tablet computer, etc.), server, dedicated switch, phone, PDA, or anyother sort of electronic or computing device. Such an electronic systemincludes various types of computer readable media and interfaces forvarious other types of computer readable media. Electronic system 1300includes a bus 1305, processing unit(s) 1310, a system memory 1325, aread-only memory 1330, a permanent storage device 1335, input devices1340, and output devices 1345.

The bus 1305 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of theelectronic system 1300. For instance, the bus 1305 communicativelyconnects the processing unit(s) 1310 with the read-only memory 1330, thesystem memory 1325, and the permanent storage device 1335.

From these various memory units, the processing unit(s) 1310 retrievesinstructions to execute and data to process in order to execute theprocesses of the invention. The processing unit(s) may be a singleprocessor or a multi-core processor in different embodiments.

The read-only-memory (ROM) 1330 stores static data and instructions thatare needed by the processing unit(s) 1310 and other modules of theelectronic system. The permanent storage device 1335, on the other hand,is a read-and-write memory device. This device is a non-volatile memoryunit that stores instructions and data even when the electronic system1300 is off. Some embodiments of the invention use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) asthe permanent storage device 1335.

Other embodiments use a removable storage device (such as a floppy disk,flash memory device, etc., and its corresponding drive) as the permanentstorage device. Like the permanent storage device 1335, the systemmemory 1325 is a read-and-write memory device. However, unlike storagedevice 1335, the system memory 1325 is a volatile read-and-write memory,such a random access memory. The system memory 1325 stores some of theinstructions and data that the processor needs at runtime. In someembodiments, the invention's processes are stored in the system memory1325, the permanent storage device 1335, and/or the read-only memory1330. From these various memory units, the processing unit(s) 1310retrieves instructions to execute and data to process in order toexecute the processes of some embodiments.

The bus 1305 also connects to the input and output devices 1340 and1345. The input devices 1340 enable the user to communicate informationand select commands to the electronic system. The input devices 1340include alphanumeric keyboards and pointing devices (also called “cursorcontrol devices”), cameras (e.g., webcams), microphones or similardevices for receiving voice commands, etc. The output devices 1345display images generated by the electronic system or otherwise outputdata. The output devices 1345 include printers and display devices, suchas cathode ray tubes (CRT) or liquid crystal displays (LCD), as well asspeakers or similar audio output devices. Some embodiments includedevices such as a touchscreen that function as both input and outputdevices.

Finally, as shown in FIG. 13, bus 1305 also couples electronic system1300 to a network 1365 through a network adapter (not shown). In thismanner, the computer can be a part of a network of computers (such as alocal area network (“LAN”), a wide area network (“WAN”), or an Intranet,or a network of networks, such as the Internet. Any or all components ofelectronic system 1300 may be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableBlu-Ray® discs, ultra density optical discs, any other optical ormagnetic media, and floppy disks. The computer-readable media may storea computer program that is executable by at least one processing unitand includes sets of instructions for performing various operations.Examples of computer programs or computer code include machine code,such as is produced by a compiler, and files including higher-level codethat are executed by a computer, an electronic component, or amicroprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some embodiments areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some embodiments, such integrated circuits executeinstructions that are stored on the circuit itself. In addition, someembodiments execute software stored in programmable logic devices(PLDs), ROM, or RAM devices.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium,” “computer readable media,” and “machinereadable medium” are entirely restricted to tangible, physical objectsthat store information in a form that is readable by a computer. Theseterms exclude any wireless signals, wired download signals, and anyother ephemeral signals.

This specification refers throughout to computational and networkenvironments that include virtual machines (VMs). However, virtualmachines are merely one example of data compute nodes (DCNs) or datacompute end nodes, also referred to as addressable nodes. DCNs mayinclude non-virtualized physical hosts, virtual machines, containersthat run on top of a host operating system without the need for ahypervisor or separate operating system, and hypervisor kernel networkinterface modules.

VMs, in some embodiments, operate with their own guest operating systemson a host using resources of the host virtualized by virtualizationsoftware (e.g., a hypervisor, virtual machine monitor, etc.). The tenant(i.e., the owner of the VM) can choose which applications to operate ontop of the guest operating system. Some containers, on the other hand,are constructs that run on top of a host operating system without theneed for a hypervisor or separate guest operating system. In someembodiments, the host operating system uses name spaces to isolate thecontainers from each other and therefore provides operating-system levelsegregation of the different groups of applications that operate withindifferent containers. This segregation is akin to the VM segregationthat is offered in hypervisor-virtualized environments that virtualizesystem hardware, and thus can be viewed as a form of virtualization thatisolates different groups of applications that operate in differentcontainers. Such containers are more lightweight than VMs.

Hypervisor kernel network interface modules, in some embodiments, is anon-VM DCN that includes a network stack with a hypervisor kernelnetwork interface and receive/transmit threads. One example of ahypervisor kernel network interface module is the vmknic module that ispart of the ESXiTM hypervisor of VMware, Inc.

It should be understood that while the specification refers to VMs, theexamples given could be any type of DCNs, including physical hosts, VMs,non-VM containers, and hypervisor kernel network interface modules. Infact, the example networks could include combinations of different typesof DCNs in some embodiments.

The term “packet” is used throughout this application to refer to acollection of bits in a particular format sent across a network. Itshould be understood that the term “packet” may be used herein to referto various formatted collections of bits that may be sent across anetwork. A few examples of such formatted collections of bits areEthernet frames, TCP segments, UDP datagrams, IP packets, etc.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. Thus, one of ordinary skill in the artwould understand that the invention is not to be limited by theforegoing illustrative details, but rather is to be defined by theappended claims.

We claim:
 1. A non-transitory machine readable medium storing a programwhich when executed by at least one processing unit associates aplurality of addresses with a logical network interface in a logicalnetwork, the logical network implemented on a set of hardware forwardingelements, the program comprising: identifying a plurality of logicaladdresses to assign to a machine of the logical network, the machineattached to a particular port of a particular hardware switch of the setof hardware switches; and modifying a set of associations stored at theparticular hardware switch to associate the particular port with theplurality of addresses.
 2. The non-transitory machine readable medium ofclaim 1, wherein the set of instructions for identifying the logicaladdresses comprises a set of instructions for receiving user input thatassigns the logical addresses to the machine.
 3. The non-transitorymachine readable medium of claim 1, wherein the logical addresses areInternet Protocol (IP) addresses.
 4. The non-transitory machine readablemedium of claim 1, wherein the set of associations are stored in a setof tables of the hardware forwarding element.
 5. The non-transitorymachine readable medium of claim 4, wherein the set of instructions formodifying the set of associations comprises sets of instructions for:generating a delimited string of the plurality of logical addresses; andstoring the delimited string in the set of tables, wherein theparticular hardware forwarding element identifies the MAC addressassociated with the plurality of logical addresses based on the storeddelimited string.
 6. The non-transitory machine readable medium of claim5, wherein the delimited string of logical addresses further comprisesat least one substring representing a range of consecutive logicaladdresses of the plurality of logical addresses.
 7. The non-transitorymachine readable medium of claim 4, wherein the set of instructions formodifying the set of associations comprises sets of instructions for:generating a string representing a range of consecutive logicaladdresses of the plurality of logical addresses; and storing thegenerated string in the set of tables, wherein the particular hardwareswitch identifies the MAC address associated with the range ofconsecutive logical addresses based on the stored generated string. 8.The non-transitory machine readable medium of claim 1, wherein the setof associations are for mapping a Media Access Control (MAC) address toa set of IP addresses.
 9. The non-transitory machine readable medium ofclaim 1, wherein the hardware forwarding element is one of a router anda switch.
 10. The non-transitory machine readable medium of claim 9,wherein the hardware forwarding element is a top-of-rack (TOR) switch ina rack.
 11. The non-transitory machine readable medium of claim 10,wherein the particular port of the TOR switch is coupled to a rackserver on the rack.
 12. The non-transitory machine readable medium ofclaim 11, wherein the set of associations further comprises bindingswith a logical switch of a plurality of logical switches that operateson the hardware switch.
 13. The non-transitory machine readable mediumof claim 1, wherein the program further comprises sets of instructionsfor: receiving an address resolution request from a machine coupled tothe hardware forwarding element; identifying a MAC address associatedwith
 14. A method for associating a plurality of addresses with alogical network interface in a logical network, the logical networkimplemented on a set of hardware forwarding elements, the methodcomprising: identifying a plurality of logical addresses to assign to amachine of the logical network, the machine attached to a particularport of a particular hardware switch of the set of hardware switches;and modifying a set of associations stored at the particular hardwareswitch to associate the particular port with the plurality of addresses.15. The method of claim 14, wherein identifying the logical addressescomprises receiving user input that assigns the logical addresses to themachine.
 16. The method of claim 14, wherein the logical addresses areInternet Protocol (IP) addresses.
 17. The method of claim 14, whereinthe set of associations are stored in a set of tables of the hardwareforwarding element.
 18. The method of claim 17, wherein modifying theset of associations comprises: generating a delimited string of theplurality of logical addresses; and storing the delimited string in theset of tables, wherein the particular hardware forwarding elementidentifies the MAC address associated with the plurality of logicaladdresses based on the stored delimited string.
 19. The method of claim18, wherein the delimited string of logical addresses further comprisesat least one substring representing a range of consecutive logicaladdresses of the plurality of logical addresses.
 20. The method of claim17, wherein modifying the set of associations comprises: generating astring representing a range of consecutive logical addresses of theplurality of logical addresses; and storing the generated string in theset of tables, wherein the particular hardware switch identifies the MACaddress associated with the range of consecutive logical addresses basedon the stored generated string.